Papers by Rasmus Kær Jørgensen
mDAPT: Multilingual Domain Adaptive Pretraining in a Single Model (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing domain-specific multilingual pretraining data is difficult to obtain due to regulations, legislation, or simply a lack of language- and domain- specific text. |
| Approach: | They propose to continue pretraining a language model on domain-specific unlabelled text . this allows for better modelling of text for downstream tasks within the domain . |
| Outcome: | The proposed approach outperforms the general multilingual model and performs close to its monolingual counterpart. |